Generally speaking, distributed computing assumes that multiple CPUs are sharing a single source of memory.

Parallel computing is another way of saying distributed computing

In general parallel computing implies multiple CPUs share the same memory.

With distributed computing, each CPU has its own memory.

In distributed computing, each computer/machine is connected to the other machines across a network.